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Abstract 

Companies purchase computer systems to make their processes more efficient through 
automation. Some academic medical centers (AMC) have purchased computer systems for their 
institutional review boards (IRB) to increase efficiency and compliance with regulations. IRB 
computer systems are expensive to purchase, deploy, and maintain. An AMC should expect a 
positive return on investment from the purchase of an IRB system. An IRB system should 
decrease the processing time for IRB applications and affect the number of IRB staff members 
necessary to adequately handle IRB processes. This study examined these and other factors at 
AMCs to determine the value of IRB computer systems. 


Introduction 

There is a lack of studies specific to the 
results of implementing a computer system 
to support IRBs. These systems are very 
expensive and challenging to implement. 
The rewards from this technology should be 
a more efficient operation that puts less of 
an administrative burden on the 
investigators utilizing IRB services. The 
outcome of this project provides evidence to 
indicate that larger IRBs are aided by 
computerized systems. 


IRBs need to efficiently process protocol 
submissions. Delays in processing 
submissions are frustrating for both the 
investigators and the research sponsors. In 
fact, for several years there has been a 
movement towards using a centralized IRB 
rather than a local IRB (Weschsler, 2007). 
One of the reasons for this preference is that 
central IRBs are often quicker to process 
protocols than local IRBs (Weschler, 2007). 
This is particularly true with multi-center 
studies where a centralized IRB would 
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provide a single, more efficient review 
rather than multiple reviews by each local 
IRB associated with the project (Food and 
Drug Administration, 2011). A study 
conducted by Whitney et al. (2008) included 
a survey of principal investigators' (PI) 
views towards IRBs. Both negative and 
positive responses towards the IRB system 
were offered by the Pis. The negative 
responses portrayed the IRB process as a 
research burden—investigators perceived 
IRB processing of research protocols as 
being slow and inefficient. Andrews et al. 
(2012) described how a local IRB can be 
made more efficient without adding 
additional cost by workflow redesign. This 
workflow redesign was done without 
adding computer systems that undoubtedly 
would have increased the cost. 

Computerized systems have been 
implemented in many industries in order to 
increase efficiency and eliminate process 
bottlenecks that can lead to employee or 
customer dissatisfaction. A few papers have 
outlined the success of implementing 
systems in research institutions to help 
track budgets and increase the productivity 
of the research administrative offices. The 
Medical University of South Carolina 
(MUSC) effectively implemented a web- 
based budgeting system (Glenn & Sampson, 
2011). The Mayo Clinic increased the quality 
of the services offered by its research 
administration offices by implementing a 
new pre-award and IRB system (Smith & 
Gronseth, 2011). A study from the Imperial 


College of London described the 
implementation of both a pre-award and 
post-award system (Rutherford & Langley, 
2007). All of these papers offer insights into 
the implementation of computer research 
administrative systems at a single 
institution. They all cite benefits of the new 
systems but are only focused on a "case 
study" regarding a particular institution's 
implementation of computerized systems. 
The studies lack the presentation of data 
from multiple institutions that supports the 
cost of purchasing and implementing new 
systems. Our study presents productivity 
data from multiple institutions that were 
used to inform the decision to acquire and 
implement an IRB computer system. 

The 2012 Metrics on Human Research 
Protection Program Performance report issued 
by the Association for the Accreditation of 
Human Research Protection Programs 
(AAHRPP) indicated that 90% of AAHRPP- 
accredited institutions reported using a 
database to track IRB protocols, while only 
40% of accredited institutions reported 
using an online system for the actual IRB 
review functions (AAHRPP, 2013). 
Additionally, the AAHRPP metrics showed 
that the "use of computer systems for the 
distribution of materials has increased 
consistently since 2009" (p. 15). A caveat to 
these statistics, however, is that the data 
come from institutions that meet the high 
standards for excellence in their Human 
Research Protection Programs required for 
AAHRPP accreditation. Therefore, these 
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data may not be applicable to all IRB 
programs in academic medical centers. 

Methods 

This study design involved a cross 
sectional survey. Data were collected via a 
12-item survey tool distributed to senior 
Institutional Review Board (IRB) 
administrators at institutions belonging to 
the University Health Consortium (UHC). 
The UHC is an organization comprised of 
120 academic medical centers and 299 of 
their affiliated hospitals. The UHC mission 
is to facilitate performance improvement for 
member institutions. Of the 120 UHC- 
member AMCs, contact information for a 
senior IRB administrator was publicly 
available for 92 institutions. A senior IRB 
administrator was defined as an individual 
within the IRB office with a job title 
equivalent to manager or above. An email 
was sent to the 92 identified IRB 
administrators with a letter of introduction 
and a link to the survey in Google Docs®. 
Two weeks after the initial contact, a 
reminder email and link to the survey were 
sent to the same cohort of identified IRB 
administrators. 


The survey was kept open for a total of 
four weeks from the initial point of contact 
to maximize response rates after each of the 
two contacts. Responses to survey questions 
were returned anonymously; however, 
respondents had the option to include 
contact information at the conclusion of the 
survey if they wished to receive a copy of 
the aggregate data as an incentive for 
participation. 

All research activities were reviewed 
and approved by the Rush University 
Medical Center Institutional Review Board. 

Results 

The data were analyzed for consistency 
and statistical significance using a standard 
statistical software package. Two survey 
responses were removed from the analyzed 
data set due to inconsistent responses. The 
two survey responses incorrectly stated or 
omitted the percentage of studies the 
organization sent to an external IRB for 
review. Table 1 provides the overall 
statistics for the survey, including the 41% 
response rate. 


Table 1 


Response Rate for Survey 


Total Surveys Sent 

92 

Total Surveys Received 

38 

Response Rate 

41.30435 
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Table 2 offers the descriptive statistics vast majority of respondents have computer 

for the data collected via the survey. The systems to help automate the IRB process. 

Table 2 


Descriptive Statistics for Survey Responses 


Variable 


N(%) 

Computer System for IRB 

Yes 

29 (76) 

No 

9(24) 

Institutions with External IRB 

Yes 

24 (63) 

No 

14 (37) 

AAHRPP Accreditation 

Yes 

27 (71) 

No 

11 (29) 


Variable 


Median (25 th 
Percentile, 75 th 
Percentile) 

Information Services FTE (with computer system) 


1.5(1, 3) 

Total Number of Active Studies at the Institution 


2100 (1275, 4000) 

Number of IRBs 


4 (2,4.25) 

Percent of Studies Sent to External IRB 


5 (2,13.75) 




Variable 


Mean + SD 

IRB Staff (FTE) 


11.64+/-7.07 

Percent Expedited Studies per Institution 


57 +/-18 

Days to Process Expedited Studies 


22.5 +/-13.5 

Percent of Studies Sent to Full IRB Board Review 


30 (24,40) 

Days to Process Full Board Studies 


40.2 +/- 20.1 


Comparison of IRB FTEs vs. IRB 
Computer Systems 

Figure 1 plots the mean IRB FTEs for 
organizations with and without a 
computerized IRB system. A one-way 
ANOVA was used to compare IRB FTEs for 


organizations with (12.224 + 6.487, n= 29) 
and without (9.778 + 8.871, n= 9) a computer 
system. The analysis indicates that there 
was no difference in the mean number of 
FTEs in organizations with or without a 
computer system (Fi, 36 = 0.819, p=0.372). 
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Figure 1. Number of IRB Full-Time Equivalents (FTE) on Organizations with and without a 
Computer System 


Comparison of IRB FTEs vs. Number 
of Studies 

A Spearman's correlation test was 
conducted on the data (see Figure 2). Test 
results indicated a statistically significant 
positive relationship between IRB FTEs and 
number of studies in this sample (r s = .831, 


pc.OOl). The effect size for this correlation is 
considered to be large and suggests that the 
correlation between IRB FTEs and number 
of studies should be considered by 
practitioners in developing their staffing 
model for IRB offices. 



Figure 2. Number of IRB FTEs vs. Number of Annual Studies 
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Comparison of Number of Studies vs. 
IRB Computer Systems 

Figure 3 depicts the difference in mean 
number of active studies for IRBs with and 
without an electronic system. The results of 
a Mann-Whitney U test (Mann-Whitney U = 


89, p = 0.154) indicated no difference in the 
median number of studies for those who 
reported not having a computer system 
[1800 (425, 3750), n= 9] and for those who 
reported having a computer system [2500 
(1600, 4300), n=29]. 
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Figure 3. Mean Number of Studies for Offices with and without a Computer System 


Comparison of Number of IRBs vs. 

IRB Computer Systems 

Figure 4 depicts the difference in mean 
number of IRBs for institutions with and 
without an electronic system. The results of 
a Mann-Whitney U test (Mann-Whitney U = 


102.5, p = 0.161) indicated no difference in 
the median number of IRBs for those who 
reported not having a computer system [3 
(1, 4.5), n= 9] and for those who reported 
having a computer system [4 (2, 4.5), n=29] 
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Figure 4. Mean Number of Studies for Offices with and without a Computer System 


Comparison of IRBs Grouped by 
Number of Studies vs. IRB Computer 
Systems 

The data were grouped into study 
categories defined by AAHRPP (see Table 
3) (AAHRPP, 2013). A Mann-Whitney test 
was performed on the categorized data to 
determine the validity of the alternative 
hypothesis that an organization with a large 
number of studies is more likely to have a 


computer system. The Mann-Whitney U test 
(Mann-Whitney U = 77, p = 0.0275), 
determined that there is a statistically 
significant difference for the categorized 
data when comparing those IRBs with and 
without a computer system. Figure 5 
graphically depicts the increase in the 
percentage of computerized IRBs as a 
function of the number of protocols 
processed per IRB. 


Table 3 


Number of Study Categories 


Number of 

Studies 

N (%) 

% with 
Computer 
Systems 

1-100 

0(0) 

0 

101-500 

4(10) 

25 

501-1,000 

3(8) 

67 

1,001-2,000 

11 (29) 

82 

2,001-4,000 

11 (29) 

82 

4,000+ 

9(24) 

89 
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Figure 5. IRBs Categorized by Protocol Volume vs. the Percent with a Computer System 


Comparison of Percent of IRBs with 
Computer Systems vs. IRBs with 
AAHRPP Accreditation 

Figure 6 compares the percentage of 
IRBs with computer systems with those that 
have AAHRPP accreditation. The chi square 


analysis for the association between having 
a computer system (yes, no) and AAHRPP 
accreditation (true, false) indicated no 
significant association between having a 
computer system and AAHRPP 
accreditation (x 2 (i) = 1.377, p<.241). 



Figure 6. Percent of IRBs with Computer Systems and Percent of IRBs with AAHRPP 
Accreditation 
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Comparison of Application Processing 
Time vs. IRB Computer Systems 

The data were also analyzed to see if the 
processing time for expedited or full board 
studies was impacted by the presence of a 
computer system. The difference in mean 
processing days for an expedited study 
(without a computer system, 23.8 + 14.8, 
n= 8; and with a computer system, 22.2 + 

13.4, n=29) was examined. The study data 
indicated that a computer system does not 
impact the mean days for an expedited 
study (independent fes = 0.289, p = 0.775). 
The difference in mean processing days for 
a full board study (without a computer 
system, 43.56 + 24.9, n=9, and with a 
computer system, 39.2 + 18.8, n= 29) was also 
examined. The study data similarly 
indicated that a computer system does not 
affect the mean days for a full board study 
(independent t36 = 0.566, p = 0.575). 

Discussion 

The null hypothesis for the study is that 
there is no difference in the mean number of 
IRB FTEs for organizations with or without 
a computer system. The results of the study 
indicate that the presence of a computer 
system has no effect on the number of IRB 
FTE staff members. However, the statistical 
analysis of the effect of number of studies 
on IRB FTE count revealed that 
organizations with more studies have a 
higher FTE count (see Figure 3). 

This study also examined whether the 
number of studies influenced an 


organization's decision to have a computer 
system. Figure 4 seems to indicate that 
organizations with more studies are more 
likely to have a computer system. 
Unfortunately, the statistical analysis did 
not support that hypothesis. However, it 
seemed that the large variation in number 
of studies [median = 2100 (1275, 4000), n=38] 
and the small sample size caused an issue 
with this statistical analysis. In order to 
smooth the variations in the data, the data 
were put into categories defined by 
AAHRPP (AAHRPP, 2013). A statistical 
analysis of these categorized data revealed 
that the number of studies processed by an 
IRB was an indicator of whether or not an 
organization used an IRB computer system. 
Figure 5 graphically depicts this result. An 
IRB that processes more studies annually is 
more likely to have an IRB computer 
system. 

The study results were limited by 
several factors. Because the survey sample 
included only IRBs that belong to the 
University Health System Consortium, the 
association of the IRB with a UHC hospital 
may influence the ability of those IRBs to 
adopt computer systems for processing 
protocols. Additionally, due to the small 
sample size from a single membership 
group, the results may not be indicative of 
larger IRB office populations. 

Therefore, future studies of the impact 
of computer systems on IRB staffing and 
efficiency should include a larger, more 
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diverse population of IRB offices. Also, the 
study should collect more detailed 
information regarding functions required of 
the computer system used by a particular 
institution. For instance, is the computer 
system used only to collect the protocol 
information or does it also support the 
workflow for processing review and 
approval of the protocol? This distinction 
would indicate the level of automation for 
the IRB office. 

Conclusion 

This study concluded that the number 
of staff in an IRB office is a function of the 


number of protocols that it processes. Also, 
the study found that IRB offices with more 
protocols are more likely to have a 
computer system. Further study is needed 
to determine if the results of this study are 
applicable to the larger population of IRB 
offices. Also, a future study should explore 
the impact of various computer system 
functions (e.g., collection of protocol 
information, automated workflows, virtual 
meetings) on IRB office staffing levels or 
protocol processing times. 
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